Compression of ipv6 addresses in a netflow directory

ABSTRACT

Modified flow keys holding compressed IPv6 addresses are stored in a flow table to improve memory utilization. The compressed IPv6 addresses are utilized to access a compression table holding the full IPv6 address, and full IPv6 address are substituted into the modified flow key to form an unmodified flow key.

BACKGROUND OF THE INVENTION

In a network switch that performs a flow based lookup, packets areclassified into “flows”, where a flow defines a particular communicationsession between two hosts and is defined by IP source and destinationaddresses and other fields from a packet header received at the networkswitch.

When a packet is processed at the network switch, its fields areextracted to form a flow key which is used by a lookup process toidentify the packet and process the packet based on the control andstate maintained for that flow. When a new flow is received a uniqueflow key is formed that includes source and destination addresses of thehosts and other fields included in packet headers. The flow keys arestored in a memory structure called a flow table. The location in theflow table that stores the flow key is accessed by a hash lookup usingan hkey which is an abbreviated version or “hash” of the flow key. Anumber of different flow keys can be hashed to the same hkey because thehkey is smaller that the flow key.

New flows are created and their flow keys are stored in a flow directoryfor later lookup. The flow directory is a memory structure that isorganized to implement a typical hash structure. A hash lookup can beperformed, for example, using a polynomial function to directly computethe memory location in which a new flow should be stored and whereexisting flows are looked up. Each location in the flow table, called abucket id, can store N (a positive integer) different flows having flowkeys which hash to the same location, where N is an architectureparameter. When any bucket is full, new flows that can potentially bemapped to that bucket will not be added, i.e., the switch runs out offlow capability at that point. When N flow keys are mapped to the samebucket then the bucket is full and no further flow keys can be mapped tothe bucket. A properly selected polynomial function can make sure abucket will not be full unless most (about 80%) of the buckets are full.

TECHNICAL FIELD

The invention relates to the technical field of efficient implementationof net flow tables in memory.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example embodiment;

FIG. 2 is a flow chart depicting an example embodiment for creating amodified flow key;

FIG. 3 is flow chart depicting an example of steps performed to generatea modified flow key;

FIG. 4 is a flow chart depicting an example of steps performed toprocess an input flow key; and

FIG. 5 is flow chart depicting an example of steps performed to retirestale IPv6 addresses.

DESCRIPTION OF EXAMPLE EMBODIMENTS Overview

In an example embodiment, a compressed Ipv6 address is stored as part ofa modified flow key in the flow table where the compressed addressincludes a hash of the Ipv6 address. The compressed Ipv6 address refersto a memory location in a compression table which holds the entire Ipv6address. The utilization of the compressed Ipv6 address allows efficientuse of the bucket in the flow table.

Description

Reference will now be made in detail to various embodiments of theinvention. Examples of these embodiments are illustrated in theaccompanying drawings. While the invention will be described inconjunction with these embodiments, it will be understood that it is notintended to limit the invention to any embodiment. On the contrary, itis intended to cover alternatives, modifications, and equivalents as maybe included within the spirit and scope of the invention as defined bythe appended claims. In the following description, numerous specificdetails are set forth in order to provide a thorough understanding ofthe various embodiments. However, the present invention may be practicedwithout some or all of these specific details. In other instances, wellknown process operations have not been described in detail in order notto unnecessarily obscure the present invention.

A first example embodiment is depicted in FIG. 1 which depicts highlevel views of a compression table 10 and a flow table 20. In thisexample, a 128-bit Ipv6 address is hashed to a 10-bit compression h-keywhich is used to access the buckets 12 of the compression table 10. Eachbucket includes eight locations for holding 128-bit Ipv6 addresses. Asdescribed in more detail below, a 10-bit compression hkey isconcatenated with a 3-bit offset to form a 13-bit compression addressthat identifies the location in the compression table holding aparticular Ipv6 address.

The flow table 20 stores a modified, or compressed, flow key having thecompression address substituted for the Ipv6 addresses included in thesource address and/or destination address fields of packets included inthe flow. The modified flow key is stored in the flow table as asubstitute for the uncompressed flow key that includes uncompressedsource and/or destination Ipv6 addresses.

This modified flow key is held in the bucket of the flow table accessedby the hash of the uncompressed flow key which includes uncompressedIPv6 source and/or destination addresses, derived from a receivedpacket.

An example of a technique for the generation of the modified flow keyfor a new flow will now be described with reference to the block diagramof FIG. 2 and the flow chart of FIG. 3. In Ipv6 address held in eitherthe source or destination address fields of a packet of a flow is inputto hash generator 40 which outputs a 10-bit compression hkey and theIPv6 address is stored in a location of the bucket indexed by the 10-bitcompression hkey. The IPv6 address is also input in parallel to eightcomparators 42(0)-(7) which compare the IPv6 address with the contentsof each of the eight storage locations in the bucket addressed by thehashed Ipv6 address. Each comparator 42 compares the contents of arespective location with the input Ipv6 address and the comparator thatfinds the matching stored Ipv6 address outputs a 1 while the othercomparators output a 0.

The outputs of the eight comparators 42 are connected in parallel to anencoder 44 which outputs a 3-bit offset address indicating which of theeight locations in the bucket stores the input Ipv6 address. The 10-bitcompressed hkey and 3-bit offset address are concatenated to form thecompression address of the location holding the input Ipv6 address andthe compression address is input to the key composition box 46. The keycomposition box 46 processes a received compression address and otherfields from the packet to form the modified flow key holding thecompression address as a replacement for the Ipv6 address.

The modified flow key is stored in the bucket of the flow table accessedby the uncompressed flow key holding the full IPv6 source and/ordestination addresses.

The processing of a received packet for an existing flow will now bedescribed with reference to the flow chart of FIG. 4. The inputuncompressed flow key of a received packet is hashed and utilized toaccess a bucket in the flow table. A compressed key is read from eachlocation of an accessed bucket in the flow table and the compressedaddresses are utilized to access the corresponding stored uncompressedIpv6 addresses from the compression table. The uncompressed Ipv6addresses are then substituted for the compressed Ipv6 addresses to formuncompressed flow keys which are compared to the input uncompressed flowkey to determine which of the flow keys stored in the bucket match theinput uncompressed flow key.

In the above-described embodiment the 128-bit Ipv6 address has beencompressed to a 13-bit address. Although the number of hosts that can beidentified has been greatly reduced, analysis of netflow trafficindicates that the 13-bit address can be effectively employed because ofthe pairing between source and destination hosts in the Net Flowparadigm. For example, although it takes 128 bits to identify IPv6hosts, not all the hosts will be part of flows that are active in theflow directory at any time. In reality, the total number of hostsrepresented in the flow directory is significantly smaller than thenumber specified by 128 bit and can be related to the total number offlows supported.

A flow is a classification of traffic between 2 hosts where N hosts cangenerate about N×N flows. However, many of the flows in the flowdirectory carry the same addresses, (A, B), (A, C), (B, A), (B, C), (C,A), (C, B), etc. Therefore, the number of addresses active at any timeis approximately the square root of the total number of active flows. Inthe above embodiment, multiples of that number are supported to takeinto account deployment scenarios where there are many-to-oneconnections.

In the above described embodiment, the compression memory can only storea small fraction of the possible Ipv6 address values. As new flows arecreated the capacity of the compression memory will be exhausted andoverflow will occur. To prevent overflow an embodiment utilizes anetflow feature of deleting stale flows to delete stale entries from thecompression table.

In this example embodiment depicted in the flow chart of FIG. 5, acounter value is maintained for each location in the compression table.When a new flow causes an IPv6 address to be stored at a location, thenthe counter for that location is incremented. As subsequent flows arereceived the counter is incremented each time a flow includes a sourceor destination address equal to the address stored in the location.Further, the counter is decremented each time a flow including the Ipv6address stored in the location is retired from the flow table. If thecounter value becomes zero then the location is indicated as free sothat a new Ipv6 address can be stored in the location.

The implementation of a netflow table utilizing the above embodimentshas many advantages. In a standard system, where both IPv4 and IPv6flows can be mapped into the same bucket, the possibility exists that asingle bucket can contain flow keys of mixed lengths, which presents anumber of problems. First, if a fixed size memory unit is defined tocarry the smaller flow key including 32-bit IPv4 addresses, then threeentries are needed to carry ipv6 addresses, resulting in waste of memorybecause more storage is used than is required by the actual size of thekey. Second, if an IPv6 flow key is stored in contiguous units the flowcreation and deletion sequence can create a distribution pattern that isfragmented, for example, when there are more than three units in abucket that are not contiguous. Third, when the flows are predominatelyIPv6 there are always bucket locations left over that will not be usedsince N is typically 2 raised to the power of a whole number and not amultiple of 3.

The invention has now been described with reference to the preferredembodiments. Alternatives and substitutions will now be apparent topersons of skill in the art. For example, the number of locations inflow table bucket and the number of bits in the compressed flow keygiven by way of example and different values may be utilized.Additionally, the principles described above may be extended to othersystems that set up connections between pairs of hosts. Accordingly, itis not intended to limit the invention except as provided by theappended claims.

1. A system comprising: a first hashing unit adapted to receive an IPv6address and to hash a received IPv6 address to a first compressed hashkey having fewer bits than the IPv6 address; a compression table, havingan address input coupled to receive compressed hash keys and havingbuckets indicated by received compressed hash keys, with a selectedbucket indicated by the first compressed hash key having a plurality oflocations adapted to hold input IPv6 addresses that are hashed to thefirst compressed hash key by the first hashing unit; a modified keycomposition unit, coupled to the first hashing unit, adapted to composea modified flow key by substituting a compressed IPv6 address, includingthe compressed hash key, for a full IPv6 address to form the modifiedflow key; a second hashing unit adapted to receive an unmodified flowkey including a full IPv6 address and to hash a received unmodified flowkey to a first full hash key; a flow table, having an address inputcoupled to receive the full hash keys and having buckets indicated bythe full hash keys, with a selected bucket indicated by the first fullhash key having a plurality of locations adapted to hold modified flowkeys corresponding to full flow keys that are hashed to the first fullhash key, with each unmodified flow key having a compressed IPv6 addresssubstituted for the full IPv6 address.
 2. The system of claim 1 furthercomprising: a plurality of comparators, each comparator having a firstinput coupled to receive a received IPv6 address, a second input coupledto receive an IPv6 address stored in a location in the selected bucketaccessed by the compressed hash key hashed from the received IPv6address, and having an output for indicating whether IPv6 addressesreceived at its inputs are the same; an encoder, having inputs coupledto the outputs of the comparators, adapted to output an offset addressindicating a location holding an IPv6 address that matches the receivedIPv6 address.
 3. The system of claim 2 where: the modified keycomposition unit concatenates the compressed hash key and the offsetaddress to form a compressed IPv6 address.
 4. A method comprising:hashing an input IPv6 address included in an input flow key of areceived packet to output a first compressed hash key having fewer bitsthan the IPv6 address; storing the input IPv6 address in one of aplurality of locations in a selected bucket of a compression table,where the selected bucket is indicated by the first compressed hash key;substituting a compressed IPv6 address, including the compressed hashkey, for the full IPv6 address in the input flow key to form a modifiedinput flow key; hashing the input flow key to output a first full hashkey; and storing the input modified flow key in a location of a selectedbucket indicated by the first full hash key in a flow table having aplurality of buckets indicated by full hash keys, with the selectedbucket having a plurality of locations adapted to hold modified flowkeys corresponding to input flow keys which hash to the first full hashkey, and with the modified flow key having a compressed IPv6 addresssubstituted for the full IPv6 address of the input flow key.
 5. Themethod of claim 4 further comprising: comparing IPv6 addresses held ineach location of the selected bucket in the compression table accessedby the input IPv6 address with the input IPv6 address to determine whichlocation stores the input IPv6 address; and encoding an offset addressindicating which location in the selected bucket holds the input IPv6address.
 6. The method of claim 5 further comprising: concatenating thecompressed hash key and offset address to form the compressed IPv6address.
 7. The method of claim 4 further comprising: maintaining acount value for a selected location in the compression table;incrementing the count value when a new flow including an IPv6 addressstored in the selected location is received; decrementing the countvalue when a flow including the IPv6 address stored in the selectedlocation is retired; and indicating that the selected location is freewhen the count value is zero.
 8. A method comprising: hashing an inputfull flow key, including a full IPv6 address, to output a first fullhash key; accessing a location of a selected bucket in a flow tablehaving buckets indicated by the full hash keys, with the selected bucketindicated by the first full hash key and having a plurality oflocations, and with each location in the plurality of locations in theselected bucket holding a modified flow key having a compressed IPv6address substituted for the full IPv6 address of the input flow key,with the modified flow keys corresponding to full flow keys that hash tothe first full flow key; for each location in the plurality of locationsin an selected bucket, utilizing the compressed IPv6 address to access alocation in a compression table holding a full uncompressed IPv6 addresscorresponding to the compressed IPv6 address; substituting a full IPv6address for a corresponding compressed IPv6 address in the modified flowkeys to form a corresponding full flow key for each modified flow keyheld in the plurality of locations to form a plurality of full flow keysfor comparison with the input full flow key.
 9. A system comprising:means for hashing an input IPv6 address included in an input flow key ofa received packet to output a first compressed hash key having fewerbits than the IPv6 address; means for storing the input IPv6 address inone of a plurality of locations in a selected bucket of a compressiontable, where the selected bucket is indicated by the first compressedhash key; means for substituting a compressed IPv6 address, includingthe compressed hash key, for the full IPv6 address in the input flow keyto form a modified input flow key; means for hashing the input flow keyto output a first full hash key; and means for storing the inputmodified flow key in a location of a selected bucket indicated by thefirst full hash key in a flow table having a plurality of bucketsindicated by full hash keys, with the selected bucket having a pluralityof locations adapted to hold modified flow keys corresponding to inputflow keys which hash to the first full hash key, and with the modifiedflow key having a compressed IPv6 address substituted for the full IPv6address of the input flow key.
 10. The system of claim 9 furthercomprising: means for comparing IPv6 addresses held in each location ofthe selected bucket in the compression table accessed by the input IPv6address with the input IPv6 address to determine which location storesthe input IPv6 address; and means for encoding an offset addressindicating which location in the selected bucket holds the input IPv6address.
 11. The system of claim 10 further comprising: means forconcatenating the compressed hash key and offset address to form thecompressed IPv6 address.
 12. The system of claim 9 further comprising:means for maintaining a count value for a selected location in thecompression table; means for incrementing the count value when a newflow including an IPv6 address stored in the selected location isreceived; means for decrementing the count value when a flow includingthe IPv6 address stored in the selected location is retired; and meansfor indicating that the selected location is free when the count valueis zero.
 13. A system comprising: means for hashing an input full flowkey, including a full IPv6 address, to output a first full hash key;means for accessing a location of a selected bucket in a flow tablehaving buckets indicated by the full hash keys, with the selected bucketindicated by the first full hash key and having a plurality oflocations, and with each location in the plurality of locations in theselected bucket holding a modified flow key having a compressed IPv6address substituted for the full IPv6 address of the input flow key,with the modified flow keys corresponding to full flow keys that hash tothe first full flow key; means for for each location in the plurality oflocations in an selected bucket, utilizing the compressed IPv6 addressto access a location in a compression table holding a full uncompressedIPv6 address corresponding to the compressed IPv6 address; means forsubstituting a full IPv6 address for a corresponding compressed IPv6address in the modified flow keys to form a corresponding full flow keyfor each modified flow key held in the plurality of locations to form aplurality of full flow keys for comparison with the input full flow key.